The present invention relates to a sound image localization device and a sound image localization method and, more particularly, to a construction for localizing a virtual sound image, in an arbitrary position, in AV (Audio, Visual) equipment.
Recently, in the fields of movie and broadcasting, multi-channel audio signals (e.g., 5.1 channel) are recorded and reproduced by using digital audio compression techniques. However, such multi-channel audio signals cannot be reproduced by an ordinary television for domestic use because the audio output of the television for domestic use is usually two or less channels. Therefore, it is expected to realize the effect of multi-channel reproduction even in such AV equipment having two-channel audio reproduction function by using the technique of sound field control or the sound image control.
FIG. 2 is a block diagram illustrating the fundamental structure of a sound image localization apparatus (sound image reproduction apparatus) according to a prior art. Initially, description will be given of a method for localizing a sound image in a position on the forward-right to the front of a listener 9 by using speakers of output units 6a and 6b which are placed in front of the listener 9. As shown in FIG. 2, the sound image localization apparatus includes a sound source 1, signal processing means 5a and 5b, and output units 6a and 6b. 
The signal source 1 is signal input means for inputting a PCM (Pulse Code Modulated) audio signal S(t). A localization angle input unit 2 is an input unit for localization information of a virtual speaker 8. A coefficient control unit 3 reads, from a coefficient memory 4, filter coefficients for localizing the virtual speaker at an angle according to the information from the localization angle input unit 2, and sets the filter coefficients in the signal processing means 5a and 5b. The signal processing means 5a is a digital filter having filter characteristics (transfer characteristics) hL(n) which are set by the coefficient control unit 3, and the signal processing means 5b is a digital filter having filter characteristics (transfer characteristics) hR(n) which are set by the coefficient control unit 3.
The output unit 6a converts the digital output supplied from the signal processing means 5a to an analog audio signal to be output. Likewise, the output unit 6b converts the digital output supplied from the signal processing means 5b to an analog audio signal to be output.
FIG. 3 is a block diagram illustrating the structure of the signal processing means 5a or 5b. The signal processing means 5a or 5b is an FIR (Finite Impulse Response) filter comprising n stages of delay elements (D) 13axcx9c13n, n+1 pieces of multipliers 14axcx9c14(n+1), and an adder 15. Input and output terminals of the respective delay elements 13 are connected with the respective multipliers 14, and the outputs from the respective multipliers 14 are added by the adder 15.
Now, the operation of the prior art sound image localization apparatus will be described with reference to FIGS. 2 and 3. In FIG. 2, a head-related transfer function between a speaker and an ear of the listener is called xe2x80x9cimpulse responsexe2x80x9d, and the value of an impulse response between the output unit 6a (speaker) and the left ear of the listener is given by h1(t). Hereinafter, impulse response is used when describing the operation in the time domain. Although the impulse response h1(t) is precisely the response in the position of the eardrum of the left ear of the listener when inputting an audio signal to the output unit 6a, measurement is performed in the position of the entrance of the external auditory miatus. The same result will be obtained even when considering the operation in the frequency domain.
Likewise, h2(t) is an impulse response between the output unit 6a and the right ear of the listener. Further, h3(t) is an impulse response between the output unit 6b and the left ear of the listener, and h4(t) is an impulse response between the output unit 6b and the right ear of the listener.
A virtual speaker 8 is a virtual sound source which is localized in a position on the forward-right to the front of the listener. Further, h5(t) is an impulse response between the virtual speaker 8 and the left ear of the listener, and h6(t) is an impulse response between the virtual speaker 8 and the right ear of the listener.
In the sound image localization apparatus so constructed, when the audio signal S(t) from the signal source 1 is output from the virtual speaker 8, the sounds reaching the left and right ears of the listener 9 are represented by the following formulae (1) and (2), respectively.
left ear: L(t)=S(t)*h5(t)xe2x80x83xe2x80x83(1)
right ear: R(t)=S(t)*h6(t)xe2x80x83xe2x80x83(2)
wherein * represents convolutional arithmetic operation. Actually, these sounds are multiplied by the speaker""s transfer function or the like, but it is ignored here to simplify the description. Alternatively, it may be assumed that the speaker""s transfer function or the like is included in h5(t) and h6(t).
Further, the impulse responses and the signal S(t) are regarded as time-wise discrete digital signals, which are represented as follows.
L(t)xe2x86x92L(n)
R(t)xe2x86x92R(n)
h5(t)xe2x86x92h5(n)
h6(t)xe2x86x92h6(n)
S(t)xe2x86x92S(n)
wherein n represents integers. When T is the sampling time, n in ( ) should be nT, precisely. However, T is omitted here.
At this time, formulae (1) and (2) are represented as the following formulae (3) and (4), respectively, and the symbol * of convolutional operation is replaced with the multiplication symbol xc3x97.
L(n)=S(n)xc3x97h5(n)xe2x80x83xe2x80x83(3)
R(n)=S(n)xc3x97h6(n)xe2x80x83xe2x80x83(4)
Likewise, when the signal S(t) is output from the output units 6a and 6b, the sound reaching the left ear of the listener is represented by the following formula (5).
Lxe2x80x2(t)=S(t)*hL(t)*h1(t)+S(t)*hR(t)*h3(t)xe2x80x83xe2x80x83(5)
When the signal S(t) is output from the output units 6a and 6b, the sound reaching the right ear of the listener is represented by the following formula (6.
Rxe2x80x2(t)=S(t)*hL(t)*h2(t)+S(t)*hR(t)*h4(t)xe2x80x83xe2x80x83(6)
When formulae (5) and (6) are represented by using (n) for the impulse responses, the following formulae (8) and (9) are obtained.
Lxe2x80x2(n)=S(n)xc3x97hL(n)xc3x97h1(n)+S(n)xc3x97hR(n)xc3x97h3(n)xe2x80x83xe2x80x83(8)
Rxe2x80x2(n)=S(n)xc3x97hL(n)xc3x97h2(n)+S(n)xc3x97hR(n)xc3x97h4(n)xe2x80x83xe2x80x83(9)
wherein hL(n) is the transfer characteristics of the signal processing means 5a, and hR(n) is the transfer characteristics of the signal processing means 5b. 
It is premised that, when the head-related transfer functions are equal, the listener hears the sounds from the same direction. This premise is generally correct. If the relationship of formula (10) is satisfied, formula (11) is established.
L(n)=L(n)xe2x80x83xe2x80x83(10)
h5(n)=hL(n)xc3x97h1(n)+hR(n)xc3x97h3(n)xe2x80x83xe2x80x83(11)
Likewise, if the relationship of formula (12) is satisfied, formula (13) is established.
R(n)=Rxe2x80x2(n)xe2x80x83xe2x80x83(12)
h6(n)=hL(n)xc3x97h2(n)+hR(n)xc3x97h4(n)xe2x80x83xe2x80x83(13)
In order to make the listener hear a predetermined sound from the position of the virtual speaker 8 by using the output units 6a and 6b, the values of hL(n) and hR(n) are decided so as to satisfy formulae (11) and (13). For example, when formulae (11) and (13) are converted into the frequency-domain expression, the convolutional operation is replaced with multiplication and, thereafter, the respective impulse responses are subjected to FFT (Fast Fourier Transform) to be transfer functions. Since the transfer functions other than that of the FIR filter are obtained by measurement, the transfer function of the FIR filter can be obtained from these two formulae.
Using hL(n) and hR(n) so decided, the signal S(n) convoluted with hL,(n) is output from the output unit 6a while the signal S(n) convoluted with hR(n) is output from the output unit 6b, whereby the listener 9 can feel the sound coming from the forward-right position even though the virtual speaker 8 does not sound actually. The FIR filter shown in FIG. 3 can localize the sound image at an arbitrary position by the signal processing described above.
Next, a description will be given of the case where the angle of the virtual speaker 8 is changed in the sound image localization apparatus.
In order to localize the virtual speaker 8 at a desired angle, the filter coefficients hL(n) and hR(n) of the signal processing means 5a and 5b must be set so as to localize the virtual speaker 8 at the desired angle. Since the filter coefficients vary according to the angle, filter coefficients of the same number as the angles to be set are required.
So, all of the filter coefficients corresponding to the respective angles to be set are stored in the coefficient memory 4. According to the angle of the virtual speaker 8, the filter coefficients for realizing the virtual speaker 8 are transferred from the coefficient memory 4 to the signal processing means 5a and 5b, followed by the sound image localization process. Thereby, the sound image localization apparatus can cope with the case where the angle of the virtual speaker 8 is changed.
The prior art apparatus and method for sound image localization are constructed as described above, and the virtual speaker can be localized with the variable angle. However, when the number of the angles of the virtual speaker 8 increases, since the coefficient memory 4 must store the filter coefficients as many as the angles, a large-capacity memory is required as the coefficient memory 4. Further, when a plurality of virtual speakers are realized in a multi-channel system, it is necessary to provide the sound image localization apparatuses as many as the virtual speakers. As the result, required computations, memory capacity, and system size are undesirably increased.
The present invention is made to solve the above-described problems and has for its object to provide a sound image localization apparatus which can realize virtual speakers of plural angles by using less parameters.
It is another object of the present invention to provide a sound image localization apparatus and a sound, image localization method which can be realized with less computational complexity and less memory capacity even in a multi-channel system.
Other objects and advantages of the invention will become apparent from the detailed description that follows. The detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the scope of the invention will be apparent to those of skill in the art from the detailed description.
According to a first aspect of the present invention, there is provided a sound image localization apparatus comprising: a signal source for outputting an audio signal; a localization angle input device for receiving an angle of a sound image to be localized; a coefficient control device for receiving sound image localization angle information from the localization angle input device, reading coefficients from a coefficient memory in accordance with the sound image localization angle information, and outputting the coefficients; first, second, and third multipliers for multiplying the audio signal output from the signal source by using first, second, and third coefficients output from the coefficient control device, respectively, and outputting the products; a first signal processing device for receiving the output from the second multiplier, and processing it by using a filter having a predetermined first frequency response; a second signal processing device for receiving the output from the second multiplier, and processing it by using a filter having a predetermined second frequency response; a first adder for receiving the output from the first multiplier and the output from the first signal processing device, and adding these outputs to output the sum; a second adder for receiving the output from the third multiplier and the output from the second signal processing device, and adding these outputs to output the sum; a first output unit for outputting the output of the first adder; and a second output unit for outputting the output of the second adder. Therefore, the virtual speaker can be localized in an arbitrary position by controlling only the coefficients of the multipliers according to the angle of the virtual speaker. As the result, a sound image localization apparatus capable of controlling the position of the virtual speaker can be realized with a coefficient memory of smaller capacity and reduced computations, as compared with those of the prior art apparatus.
According to a second aspect of the present invention, there is provided a sound image localization apparatus comprising: a signal source for outputting an audio signal; a localization angle input device for receiving an angle of a sound image to be localized; a coefficient control device for receiving sound image localization angle information from the localization angle input means, reading coefficients from a coefficient memory in accordance with the sound image localization angle information, and outputting the coefficients; first, second, and third multipliers for multiplying the audio signal output from the signal source by using first, second, and third coefficients output from the coefficient control device, respectively, and outputting the products; a signal processing device for receiving the output from the second multiplier, and processing it by using a filter having a predetermined frequency response; an adder for receiving the output from the third multiplier and the output from the signal processing device, and adding these outputs to output the sum; a first output unit for outputting the output of the first multiplier; and a second output unit for outputting the output of the adder. Therefore, the virtual speaker can be localized in an arbitrary position by controlling only the coefficients of the multipliers according to the angle of the virtual speaker. As the result, a sound image localization apparatus capable of controlling the position of the virtual speaker can be realized with a coefficient memory of smaller capacity and reduced computations, as compared with those of the prior art apparatus. Further, the construction of the apparatus can be simplified.
According to a third aspect of the present invention, there is provided a sound image localization apparatus comprising: a plurality of signal sources for outputting audio signals; a localization angle input device for receiving an angle of a sound image to be localized; a coefficient control device for receiving sound image localization angle information from the localization angle input device, reading coefficients from a coefficient memory in accordance with the sound image localization angle information, and outputting the coefficients; a plurality of signal input units provided correspondingly to the respective signal sources, each input unit having first, second, and third multipliers for multiplying the audio signal output from the corresponding signal source by using first, second, and third coefficients from the coefficient control device, respectively, and outputting the products; a first adder for summing all of the outputs from the first multipliers of the input units; a second adder for summing all of the outputs from the second multipliers of the input units; a third adder for summing all of the outputs from the third multipliers of the input units; a first signal processing device for receiving the output from the second adder, and processing it by using a filter having a predetermined first frequency response; a second signal processing device for receiving the output from the second adder, and processing it by using a filter having a predetermined second frequency response; a fourth adder for receiving the output from the first adder and the output from the first signal processing device, and adding these signals to output the sum; a fifth adder for receiving the output from the third multiplier and the output from the second signal processing device, and adding these signals to output the sum; a first output unit for outputting the output of the fourth adder; and a second output unit for outputting the output of the fifth adder. Therefore, the virtual speaker can be localized in an arbitrary position. As the result, even in a multi-channel system, a sound image localization apparatus capable of controlling the position of the virtual speaker can be realized with a coefficient memory of smaller capacity and reduced computations as compared with those of the prior art apparatus.
According to a fourth aspect of the present invention, there is provided a sound image localization apparatus comprising: a plurality of signal sources for outputting audio signals; a localization angle input device for receiving an angle of a sound image to be localized; a coefficient control device for receiving sound image localization angle information from the localization angle input device, reading coefficients from a coefficient memory in accordance with the sound image localization angle information, and outputting the coefficients; signal input units provided corresponding to the respective signal sources, each input unit having first, second, and third multipliers for multiplying the audio signal output from the corresponding signal source by using first, second, and third coefficients output from the coefficient control device, respectively, and outputting the products; a first adder for summing all of the outputs from the first multipliers of the input units; a second adder for summing all of the outputs from the second multipliers of the input units; a third adder for summing all of the outputs from the third multipliers of the input units; a signal processing device for receiving the output from the second adder, and processing it by using a filter having a predetermined frequency response; a fourth adder for receiving the output from the third multiplier and the output from the signal processing means, and adding these signals to output the sum; a first output unit for outputting the output of the first adder; and a second output unit for outputting the output of the fourth adder. Therefore, the virtual speaker can be localized in an arbitrary position. As the result, even in a multi-channel system, a sound image localization apparatus capable of controlling the position of the virtual speaker can be realized with a coefficient memory of smaller capacity and reduced computations, as compared with those of the prior art apparatus. Further, the construction of the apparatus can be simplified.
According to a fifth aspect of the present invention, any of the above-described sound image localization apparatuses further comprises a filter device for receiving filter coefficients of the predetermined frequency response from the coefficient control device, and processing the signal from the signal source. The first, second, and third multipliers multiply, not the output signal from the signal source, but the output from the filter device by using the first, second, and third coefficients from the coefficient control device, respectively. Therefore, a sound image localization apparatus capable of controlling the position of the virtual speaker and having a sound quality as high as that of the prior art apparatus, can be realized with a coefficient memory of smaller capacity and reduced computations, as compared with those of the prior art apparatus.